3.2 Service Orchestration One Pager

(Based on template version 1.92)

1. Introduction

1.1. Project/Component Working Name:

GlassFish 3.2 Service Orchestration (Service Metadata, Provisioning and Association)

1.2. Name(s) and e-mail address of Document Author(s)/Supplier:

Bhavanishankara Sapaliga: bhavanishankar@java.net
Jagadish Ramu: jr158900@java.net
Sanjeeb Sahoo: ss141213@java.net
Sivakumar Thyagarajan: sivakumart@java.net

1.3. Date of This Document:

04/25/10

2. Project Summary

2.1. Project Description:

Single-click application provisioning and deployment of a PaaS application through service dependency discovery, service provisioning and service association.

See below for a longer, more detailed technical description.

2.2. Risks and Assumptions:

The Service Orchestration layer of GlassFish 3.2 is dependent on the following set of new 3.2 infrastructure subsystems:

• the Virtualization/IaaS (using the Template catalog and the IMS API for provisioning VMs)
• PaaS Deployment(for integration of the Orchestrator into the Deployment SPI) and
• Administration/Management areas.

The Deployment and the Elasticity teams also depend on the Service Orchestrator layer.

Management of dependencies between these components under active development is a risk. Any delay in the delivery of the depended features will affect the delivery of this feature.

3. Problem Summary

3.1. Problem Area:

The Wikipedia definition of PaaS states the following as a key characteristic of a PaaS offering

"... facilitate deployment of applications without the cost and complexity of ... managing the underlying hardware and software and provisioning hosting capabilities, providing all of the facilities required to support the complete life cycle of building and delivering web applications and services".

The current GlassFish deployment model assumes that the deployer also provisions the various dependent services of an application. To support PaaS deployment scenarios, it is imperative for GlassFish 3.2 to provide a simplified application provisioning and deployment interface to users, and let the runtime handle the discovery of service dependencies of a Java EE application, provision services and associate the application's service references to the provisioned services.

3.2. Justification:

• Eases the deployment experience in PaaS usecases (single click application provisioning and deployment)
• Reduces cost and complexity of managing underlying hardware and software for the deployer
• Ensures effective utilization of resources through virtualization and/or use of compute/storage resources on-demand, auto-scaling using elasticity rules.

4. Technical Description:

4.1. Details:

A PaaS deployment of a Java EE application is handled through an Orchestrator module running in DAS. This module enables single-click application provisioning and deployment of a PaaS application through service dependency discovery, service provisioning and service association.

Terminology
Any entity that is created, provisioned, managed, and monitored by the PAAS runtime/infrastructure is a Service. Examples of Services could be Java EE(GlassFish clusters and standalone server instances), JMS providers(Open MQ broker or broker clusters), databases (Derby, MySQL) etc. A Service Provider provides an implementation of a Service. A Service implementation is an implementation of a particular class of Services, called ServiceType. Standard ServiceTypes that could be supported by GlassFish 3.2 could be RDBMS, JMS, LoadBalancer. There could be multiple Service implementations of a ServiceType concurrently deployed in a PAAS runtime. For instance, there could be Derby and MySQL database plugins for the RDBMS ServiceType available at the same time in the PAAS runtime.

Service Metadata
An application specifies its dependencies to Services using Service Requirements. Service Requirements usually consists of the following:

• Service Definition: Information(metadata) used to provision an instance of the Service.
• Service References (or Service Bindings): This is information provided by the application to indicate its dependency on a Service.
  In short, the Service Definition metadata is used to provision an instance of the Service at runtime, and the Service Reference metadata is used to bind an application or an application component's dependencies to a Provisioned Service.

A mechanism to specify optional deployment-time services meta-data, glassfish-services.xml (suggested name – final name TBD, see here for more information on the current proposed schema and sample specification) would be introduced. This descriptor could be used by the Deployer to include service definitions of an application. The Service bindings/service references of an application would be specified in glassfish-resources.xml (see here for more information). The Service Reference could be a reference to an application-scoped provisioned service or global provisioned service or external service. This descriptor would also be used by the Orchestrator to map the generic service meta-data to a Service Template in an IMS template catalog.

Service Dependency Discovery
The Service References for an application would be discovered through the following means:

• Implicitly derived through references to external resources in standard deployment-descriptors/annotations. For example, a Java EE web application (WAR) could explicitly specify a Service Reference to a RDBMS ServiceType by specifying a <resource-ref> whose <res-type> is javax.sql.DataSource in its web.xml. The Orchestrator and plugin could use this information to derive a service reference for this application to a RDBMS ServiceType.
• Explicit Declaration of Service references in glassfish-services.xml
• Discovered through other means. These are references which are not encoded in the application bits. For instance, an application deployed as a WAR/EAR, by virtue of its packaging, implies a service reference to a Java EE ServiceType

Services
The Orchestrator must support binding of an application to the following types of Services:

• External services: these are not provisioned by the orchestrator, but created and managed by an external user. Their lifecycle is not tied to an application and is not managed by the orchestrator. This is useful in scenarios where a user has existing DB/MoM investments and they want to reuse their existing setup for their application.
• Global Provisioned services: Services that are created by a DAS administrator that are provisioned and managed by the orchestrator. These services can be shared by multiple applications in the domain and the service lifecycle is not tied to a single application. This type of service is useful, for example, when multiple applications in a domain want to share the same database.
• Application-Scoped Provisioned services (These are services that are automatically provisioned by the orchestrator during PaaS-style deployment of the application, and their life-cycle is tied to the application's lifecycle.)

The list of ServiceTypes and Services supported for this release are RDBMS(JavaDB, MySQL), HTTPLoadBalancer (potentially 3 plugins: Native LB, Java LB, Apache mod_jk - exact set of supported LB configurations is TBD) and JMS(OpenMQ).

The orchestrator must not assume a fixed set of ServiceTypes and a limited set of Service Providers per ServiceType. A plugin architecture (Service Provider) must allow extending the orchestrator to support provisioning and association of new ServiceTypes and Service Providers for a ServiceType. SPIs to be able to add/remove the Plugins into the Orchestrator. The SPI will also define contracts for service life-cycle related activities viz., create, start, stop, destroy. For this release, these SPIs will be private to GlassFish. The various Service integration teams (container teams) would deliver the plugins. For instance, the MQ integration team delivers the OpenMQ plugin, the JDBC team delivers the JavaDB and MySQL plugins, and the LB team delivers the various load balancer plugins.

All the administration operations/commands on a managed entity(such as a cluster provisioned by the orchestrator) must be intercepted by the Orchestrator. The orchestrator will decide whether to allow or disallow these operations. (For example, orchestrator might not allow 'asadmin stop-cluster' on a managed cluster). Moreover, orchestrator might need to perform some additional tasks (eg: for book-keeping purposes) during the command execution on a managed object.

Orchestration Flow
When a PaaS Java EE archive(a WAR/EAR with additional optional metadata specified through glassfish-services.xml) is deployed in a cloud environment, see GlassFish 3.2 Deployment OnePager, the Orchestrator performs the following deployment actions:

• Service Dependency Discovery: Scans the application(standard deployment descriptors, glassfish-services.xml) to get effective Service Requirements for this application archive.
• Service Provisioning:
  • Automatic Service Provisioning: Provision all the required dependent services. The application could come up with full/partial/no PaaS meta-data.
    During Service Provisioning, the Orchestrator must match the service requirements of the application to a Service Template. This template would then be used to provision and realize the Service through the IMS API. These matching rules are still a work in progress. One simple rule that has been identified is: If the application requires a RDBMS service and the service definition is not specified in the PaaS meta-data or the service definition does not explicit point to a particular Service Provider(say MySQL), then by default the Orchestrator will select a "default.html" template (e.g: Derby service template).
• Service Association/Binding: Automatically creates the necessary artifacts in GlassFish(for instance, create the necessary pools/resources etc) to bind an application to the services provisioned for the application.
• Application Deployment: Deploy and load the application.

PaaS Deployment Scenarios

• Deployment: Provisioning of dependent services during application deployment must be atomic. If a subset of dependent services cannot be provisioned, the deployment must fail and the state of the system must roll-back to the state prior to the current deployment. When multiple plugins can handle particular service requirement and a particular service provider can not be matched then deployment must fail. The Orchestrator must also be able to discover and provision all the required services during the deployment of a vanilla Java EE archive (ie an application that is PaaS unaware, doesn't contain any services meta-data).
• Undeployment: Automatically decommission all the application-scoped provisioned services of an application during application undeployment.
• Enable/Disable: Pause/Stop all the application-scoped provisioned services when an application is disabled. Similarly, start all the provisioned services when the application is enabled.
• Redeployment: Support redeployment of the application. Add a "--retain" redeployment option to re-use the previously provisioned services instead of re-provisioning them again. Default value of --retain is true.
• Service decomissioning during domain deletion: Decommission all the provisioned services (application scoped and global/shared) during domain deletion.

4.2. Bug/RFE Number(s):

The Umbrella RFE is GLASSFISH-16402. The related issues are in the dependency "Issue Links" section of that issue.

4.3. In Scope:

Features discussed above.

4.4. Out of Scope:

• Advanced redeployment scenarios
  • Update and Upgrade of Service References
• Extensible Service Provider mechanism
• Elastic scaling of Services
• Allow a Provisioned Service's lifecycle to be managed by a PaaS System Adminstrator
• Support mapping to external PaaS Service Offerings (for instance Amazon RDS)
• Support for other ServiceTypes (Caching - Coherence, Mail, Queueing, NoSQL)
• Advanced Service Matching Heuristics
• Service elevation ie., upgrade an application-scoped service to shared-service when needed.

For an extended list, see the Non-Requirements/Out of Scope section here.

4.5. Interfaces:

4.5.1 Public Interfaces

The definition for this interfaces is still a work in progress and hence should be considered as draft for now.

• Interface: glassfish-services.xml
• Comment: services meta-data for an application. A proposed schema and discussion is here.

4.5.2 Private Interfaces

• Interface: domain.xml config changes TBD.

4.5.3 Deprecated/Removed Interfaces:

Not applicable

4.6. Doc Impact:

New man pages for the new command. Modifications to Application Deployment Guide, Application Development Guide, a new PaaS Guide or modify Administrators Guide, Troubleshooting Guide to cover the new PaaS deployment and Service Orchestration features.

4.7. Admin/Config Impact:

• Interface: External Services management

  create-external-service [--default=false] [--force=false] --type servicetype
  [--property (name=value)[:name=value]*] servicename}}
  

Creates an external service reference config element in domain.xml and creates a global shared connection pool(if relevant) pointing to that external service for use by applications. The supported ServiceTypes are RDBMS, JMS, and HTTPLB. The name-value property pair provides the necessary configuration to connect to the external service. For instance, if the ServiceType is RDBMS, the specified properties must be database URL, username, password, driver location(??).

list-external-services [--type servicetype]

Lists all external services registered in this domain, optionally filtered by specified ServiceType.

delete-external-service servicename

Deletes an external service reference. Ensures that there is no active usage of the deleted service reference before deleting it.

• Interface: Shared Provisioned Services management

  create-shared-service [--default=false] [--force=false] [--init-type=EAGER|LAZY] --template templatename
  [--property (name=value)[:name=value]*] servicename
  

Provisions a shared service through the Orchestrator and makes it available for use by other applications in the domain. The command creates a service reference config element in domain.xml, provisions the Service if the init-type is EAGER and creates a global shared connection pool(if relevant) pointing to that provisioned service for use by applications. If init-type is LAZY, the service provisioning and resource creation step is performed when the first application bound to that shared service is deployed/loaded. The name-value property pair provides any additional optional configuration of the provisioned service.

list-shared-services

Lists all shared services registered in this domain.

delete-shared-service servicename

Decommissions a shared provisioned service and deletes a shared service config element in domain.xml. Ensures that there is no active usage of the deleted service reference before deleting it.

• Interface: PaaS Deployment. See Deployment OnePager

• Interface: GUI. See Admin Console OnePager

4.8. HA Impact:

No Impact.

4.9. I18N/L10N Impact:

No Impact.

4.10. Packaging, Delivery & Upgrade:

4.10.1. Packaging

A new Orchestrator package.

4.10.2. Delivery

The Orchestrator would only be part of the PaaS/Cloud distrbution/environment and would only run in DAS.

4.10.3. Upgrade and Migration:

Not applicable.

4.11. Security Impact:

In a virtualized hybrid deployment environment, where a DB might lie within a firewall and the provisioned GlassFish cluster may reside outside, the virtualization layer must be configured such that DB ports would need to be accessible to the provisioned instances.

4.12. Compatibility Impact

No Impact.

4.13. Dependencies:

4.13.1 Internal Dependencies

PaaS deployment
GlassFish 3.2 Administration
IaaS Managment Service(IMS)/Virtualization

4.13.2 External Dependencies

Not applicable.

4.14. Testing Impact:

• New developer tests
• Tests to use a mock IaaS layer or work with developer mode virtualization providers such as VirtualBox/KVM.

5. Reference Documents:

• Deployment OnePager
• IMSPlugins
• Virtualization
• Admin Console OnePager
• Elasticity OnePager
• GlassFish 3.2 Service Orchestration Technical requirements page

6. Schedule:

6.1. Projected Availability:

1. Is the NativeLB service going to be supported via a default template that has native-lb pre-installed on a webserver ? 2. Under "Service Dependency Discovery" it is mentioned that the third means of discovering a dependency is through "other means". Can you give an example ? Is this through resource.xml packaged in the app ? Are there any other examples ? 3. Section "PaaS Deployment Scenarios", first bullet item "Deployment" notes that - "When multiple plugins can handle particular service requirement and a particular service provider can not be matched then deployment must fail." Question: In the case mentioned, is deployment failed only if multiple plugins are available for that service requirement ? What if there is just one plugin (i think MQ is the current example ?) that satisfies a service requirement but the service characteristics specified in the glassfish-services.xml do not match any template. Is the plan then to still fail deployment ? 4. How different are EAGER and LAZY init options of create-shared-service from user perspective ? In what kind of a scenario would a user prefer one to the other ? Do both need to be supported ? Is it possible to support the EAGER init option ? Wouldn't the created shared service/resource need to refer a target cluster, while the cluster is created much after the create-shared-service command is used ? Posted by vr143562 at May 01, 2011 05:18

How does the user know the service url or host/port after deploying an app. How is this available for a webapp and ejb (for  appclient or standalone). Posted by sonymanuel at May 01, 2011 11:00

We should use glassfish-resources.xml to define service bindings (aka service-references). e.g., a jdbc-connection-pool can be used to represent a binding to a RDBMS, a connector-connection-pool for a JMS system. Posted by sahoo_sk at May 01, 2011 23:00

Responses from Bhavani and Jagadish : Response to question-2 : Service dependency discovery paragraph categorizes the various modes by which dependencies could be discovered. This particular case states that by virtue of an user deploying a .war/.ear, a JavaEE service provider (runtime) is required. Response to question-3 : Deployment will fail when there are multiple plugins that matches the service-requirement, when there is no plugin that could match the service requirement. Service characteristics is also part of service-requirement and hence they need to be satisfied by a plugin, failing which deployment will fail. Response to question-4 : Consider the case where a shared-database-service is created with "lazy-init"=true. No DB instance/ resource, pool is created (made available)  When the first application is deployed that has a service requirement for the shared-database-service created above, then the database service would be provisioned, connection-pool would be created which could be used by the application.  Posted by jagadish at May 02, 2011 01:20

Refer section 5.3.a in the Technical requirements document (link below) that lists the need for showing all services used by an application : http://wikis.sun.com/display/GlassFish/3.2+Service+Orchestration+Technical+Requirements Posted by jagadish at May 02, 2011 01:23

Yes, there will be load-balancer templates provided for 3.2. This will include templates created using our GlassFish load-balancer plugin pre-installed on web server. Posted by kshitiz_s at May 02, 2011 02:04

Yes, there will be load-balancer templates provided for 3.2. This will include templates created using our GlassFish load-balancer plugin pre-installed on web server. Posted by kshitiz_s at May 02, 2011 04:07